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Foreword 



This report presents information collected from 2- and 4-year degree-granting 
postsecondary institutions that were sampled for the 2004 National Study of Postsecondary 
Faculty (NSOPF:04). NSOPF:04 surveyed institutions as well as faculty and instructional staff 
who were employed in these institutions in the United States in fall 2003. This report describes 
recent hiring and retirement patterns as well as tenure -related changes and actions taken by 
institutions. 

NSOPF:04 is the fourth in a series of data collections on postsecondary faculty conducted 
by the National Center for Education Statistics (NCES). The first was conducted in 1987-88, the 
second in 1992-93, and the third in 1998-99. Readers should consult appendix B of this report 
for more technical information about NSOPF:04. 

The estimates presented in this report were produced using the NCES Data Analysis 
System (DAS), a web-based table-generating application that provides the public with direct, 
free access to NSOPF data as well as other postsecondary datasets collected by NCES. The DAS 
produces the design-adjusted standard errors necessary for testing the statistical significance of 
differences in the estimates (all differences reported in the text are statistically significant at the 
.05 level). Public-access data files and descriptive reports for this and other postsecondary 
datasets collected by NCES are available at http://nces.ed.gov/das . 
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Introduction 



This report describes selected polices and practices at public and private not-for-profit 
postsecondary institutions that offered an associate’s or higher degree in fall 2003 and 
participated in federal Title IV student aid programs. The 2004 National Study of Postsecondary 
Faculty (NSOPF:04) included a survey of postsecondary faculty and instructional staff as well as 
a survey about postsecondary institutions completed by a survey coordinator at each sampled 
institution. The data in this report are from the institution component of NSOPF:04. The tables 
present data on institutional policies and practices such as faculty attrition and recent tenure- 
related changes and actions, with selected results described in the next section. 1 A glossary of 
variables in appendix A provides details about institution type, which is derived from the 2000 
Carnegie Classification, and other variables used in this report. All comparisons made in the text 
were tested using Student’s t statistic, and all differences cited were statistically significant at the 
.05 level. Standard errors for estimates in this report are available at 

http://nces .ed. gov/das/library/reports .asp . The technical notes in appendix B supply additional 
information about the methodology of the data collection, file preparation, and analysis. 

NSOPF:04 is the fourth in a series of data collections describing postsecondary faculty and 
instructional staff at 2- and 4-year degree-granting institutions. Previous administrations of 
NSOPF, including the institution survey, took place in 1987-88, 1992-93, and 1998-99. The 
responses represent approximately 3,380 public and private not-for-profit, Title IV-participating, 
degree-granting institutions in the 50 states and the District of Columbia. 2 Information about 
institutions in NSOPF:04 is based on survey data from a nationally representative sample of 
about 1,070 eligible institutions, using a web-based questionnaire that was either self- 
administered or conducted via telephone with a trained interviewer. About 920 institutions 
completed the survey, resulting in a weighted response rate of 84 percent. 3 



1 These data are similar to those presented in Institutional Policies and Practices: Results From the 1999 National Study of 
Postsecondary Faculty, Institution Survey (Berger, Kirshstein, and Rowe 2001) for fall 1998, except that report uses the 1994 
Carnegie Classification; whereas the tables presented here use the 2000 Carnegie Classification. 

2 Throughout this report, institution counts are rounded to the nearest 10 to protect the confidentiality of responding institutions. 
Percentages cited are based on the original unrounded numbers. 

3 See technical notes for more information on response rates and nonresponse bias analysis. 
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Selected Results 



• Thirty-two percent of postsecondary institutions had unions that represented at least some 
of their faculty in fall 2003 (figure 1). Public institutions were more likely than private 
institutions to have their faculty represented by a union. This is consistent with some 
legal developments that may limit collective bargaining by faculty in private institutions 
(Smallwood 2005). 4 

• In fall 2003, public institutions granting doctoral, master’s, and associate’s degrees 
employed two-thirds (25, 12, and 31 percent, respectively) of the nation’s faculty and 
enrolled three-quarters (23, 15, and 39 percent, respectively) of the nation’s 
postsecondary students (table 1). 

• Institutions reported that in fall 2003, some 8 percent of all full-time faculty were hired 
the previous year from outside the institution (table 2). Fourteen percent of all new full- 
time faculty were previously employed in part-time positions at the institution. Public 
associate’s institutions were particularly likely to hire part-time faculty into full-time 
positions at the institution. About one-fifth (21 percent) of all new full-time hires at 
public associate’s institutions had previously been employed as part-time faculty at the 
institution, compared with 7-9 percent at public and private master’s and doctoral 
institutions. 

• Institutions reported that in fall 2003, some 7 percent of all full-time faculty had left their 
institutions during the previous year (table 3). Thirty-six percent of those who left retired, 
and the remaining 64 percent left for a variety of other reasons. Some of these departures 
may have been related to actions taken by the institutions. 

• At least 90 percent of all doctoral or master’s institutions had tenure systems (table 4). 
They were less common at public associate’s (64 percent) institutions. Most institutions 
(71 percent) had tenure systems in place in fall 2003. 

• The majority (82 percent) of institutions that had tenure systems in fall 2003 limited the 
number of years that a faculty member could remain on tenure track without receiving 
tenure (table 4). About 64 percent of institutions limit time on tenure track to 6 or 7 years. 
Fifty-three percent of public associate’s institutions with tenure systems limited the 
maximum time that faculty could be on tenure track without receiving tenure to less than 
5 years. 

• Between 1998 and 2003, two-thirds (67 percent) of all institutions took at least one action 
related to tenure (table 5). The most common action taken was to offer early or phased 
retirement to tenured faculty members. This action was taken by a larger proportion of 
institutions (52 percent) than other tenure-related actions (10-15 percent) during the past 



4 The Yeshiva Decision of the United States Supreme Court in 1980 limited the ability of faculty in the private sector to unionize. 
This decision applies only to private sector schools and not to public institutions. 



3 





Selected Results 



5 years. Other tenure-related actions include replacing some tenured faculty with full- 
time faculty on fixed-term contracts, changing policy for granting tenure, making 
standards more stringent for granting tenure, and downsizing tenured faculty. 
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Figures and Tables 



Figure 1. Percentage of institutions with full- or part-time faculty represented by a union, by institution 
type: Fall 2003 
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2 
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1 All public and private not-for-profit Title IV degree-granting institutions in the 50 states and the District of Columbia. 

“ Doctoral includes research/doctoral institutions and specialized medical schools and medical centers as classified by the 2000 
Carnegie Classification. 

3 Includes public baccalaureate, private not-for-profit associate’s, and other specialized institutions except medical schools and 
medical centers. 

NOTE: Standard error tables are available at http://nces.ed.gov/das/librarv/rcports.asp . 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary 
Faculty (NSOPF:04). 
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Table 1. Percentage distribution of postsecondary education institutions, faculty, and students enrolled, 
by institution type: Fall 2003 



Institution type 


Institutions 


Total 


Faculty 

Full-time 


Part-time 


Students 

enrolled 


All institutions 1 


100.0 


100.0 


100.0 


100.0 


100.0 


Public doctoraP 


6.3 


24.8 


36.1 


12.0 


22.9 


2 

Private not-for-profit doctoral 


3.6 


11.3 


14.3 


8.0 


6.5 


Public master’s 


8.9 


12.1 


13.9 


10.1 


15.1 


Private not-for-profit master’s 


10.3 


9.3 


6.8 


12.0 


7.3 


Private not-for-profit baccalaureate 


15.7 


6.0 


6.5 


5.6 


4.6 


Public associate’s 


32.3 


31.1 


17.7 


46.2 


39.1 


Other 3 


22.9 


5.4 


4.7 


6.2 


4.5 



1 All public and private not-for-profit Title IV degree-granting institutions in the 50 states and the District of Columbia. 

2 Doctoral includes research/doctoral institutions and specialized medical schools and medical centers as classified by the 2000 
Carnegie Classification. 

3 Includes public baccalaureate, private not-for-profit associate’s, and other specialized institutions except medical schools and 
medical centers. 

NOTE: Faculty includes all faculty and instructional staff. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Integrated Postsecondary Education Data 
System (IPEDS), 2004. 
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Table 2. Percentage of newly hired full-time faculty and percentage of new full-time faculty who were 
part-time faculty at that same institution in November 2002, by institution type: Fall 2003 



Institution type 


Percent of full-time faculty 
hired within the past 
year from outside 
the institution 


Percent of new full-time 
faculty who were 
previously part time 
at the same institution 


All institutions 1 


8.4 


14.3 


Public doctoraP 


8.5 


9.2 


Private not-for-profit doctoral" 


8.9 


7.7 


Public master’s 


8.6 


6.7 


Private not-for-profit master’s 


8.3 


7.4 


Private not-for-profit baccalaureate 


9.2 


12.5 


Public associate’s 


6.4 


21.3 


Other 3 


10.4 


13.8 



1 All public and private not-for-profit Title IV degree-granting institutions in the 50 states and the District of Columbia. 

2 Doctoral includes research/doctoral institutions and specialized medical schools and medical centers as classified by the 2000 
Carnegie Classification. 

3 Includes public baccalaureate, private not-for-profit associate’s, and other specialized institutions except medical schools and 
medical centers. 

NOTE: Faculty includes all faculty and instructional staff. Standard error tables are available at 
http://nccs.cd.gov/das/librarv/rcports.asp . 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary 
Faculty (NSOPF:04). 
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Table 3. Percentage of full-time faculty who left institutions between fall 2002 and fall 2003 and percentage 
distribution by reason for leaving, by institution type: Fall 2003 



Institution type 



Full-time faculty 
who left 



Of those who left, reason for leaving 
Retired Other reason 



All institutions 


6.9 


35.6 


64.4 


Public doctoraP 


8.0 


29.9 


70.1 


Private not-for-profit doctoral" 


7.5 


24.0 


76.0 


Public master’s 


8.0 


43.2 


56.8 


Private not-for-profit master’s 


6.0 


31.0 


69.0 


Private not-for-profit baccalaureate 


7.8 


16.8 


83.3 


Public associate’s 


6.4 


46.1 


53.9 


Other 3 


6.8 


36.9 


63.1 



1 All public and private not-for-profit Title IV degree-granting institutions in the 50 states and the District of Columbia. 

2 Doctoral includes research/doctoral institutions and specialized medical schools and medical centers as classified by the 2000 
Carnegie Classification. 

3 Includes public baccalaureate, private not-for-profit associate’s, and other specialized institutions except medical schools and 
medical centers. 

NOTE: Faculty includes all faculty and instructional staff. Standard error tables are available at 
http://nccs.cd.gov/das/librarv/rcports.asp . 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary 
Faculty (NSOPF:04). 
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Table 4. Percentage of institutions with tenure systems, percentage that limit time on tenure track, and 
percentage distribution of the maximum number of years on tenure track without tenure, by 
institution type: Fall 2003 



Institutions 

that limit Maximum number of years on tenure track 

Institutions time on Less More 

with tenure tenure than 5 than 7 



Institution type 


systems 


track 


years 


5 years 


6 years 


7 years 


years 


All institutions 1 


71.5 


82.5 


21.4 


10.3 


32.6 


31.4 


4.4 


Public doctoral” 


100.0 


92.7 


# 


1.2 


40.9 


48.2 


9.8 


Private not-for-profit doctoral” 


92.1 


89.3 


# 


3.6 


36.1 


38.6 


21.7 


Public master’s 


97.9 


98.7 


# 


22.2 


37.2 


40.2 


0.4 


Private not-for-profit master’s 


92.8 


83.3 


# 


8.5 


48.2 


33.5 


9.8 


Private not-for-profit baccalaureate 


83.8 


89.8 


0.6 


# 


59.2 


36.0 


4.2 


Public associate’s 


63.9 


84.5 


52.6 


19.9 


9.8 


16.5 


1.2 


Other 3 


50.3 


58.1 


46.8 


3.2 


18.0 


31.6 


0.4 



# Rounds to zero. 

1 All public and private not-for-profit Title IV degree-granting institutions in the 50 states and the District of Columbia. 

” Doctoral includes research/doctoral institutions and specialized medical schools and medical centers as classified by the 2000 
Carnegie Classification. 

3 Includes public baccalaureate, private not-for-profit associate’s, and other specialized institutions except medical schools and 
medical centers. 

NOTE: Detail may not sum to totals because of rounding. Standard error tables are available at 
http://iiccs.cd.gov/das/library/rcports.asp . 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary 
Faculty (NSOPF:04). 
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Table 5. Percentage of institutions that took specific actions related to tenure during the past 5 years, by 
institution type: Fall 2003 









Actions related to tenure 1 




Institution type 


Took at 
least one 
action 
related to 
tenure 


Offered 
early or 
phased 
retirement 
to tenured 
faculty 


Replaced 
some 
tenured 
faculty with 
full-time 
faculty on 
fixed-term 
contracts 


Changed 
policy for 
granting 
tenure 


Made 
standards 
more 
stringent 
for granting 
tenure 


Downsized 

tenured 

faculty 2 


All institutions' 


66.6 


51.8 


14.2 


12.0 


10.4 


14.6 


Public doctoral 4 


72.3 


55.4 


28.3 


14.7 


17.0 


17.0 


Private not-for-profit doctoral 4 


77.4 


59.1 


14.0 


17.2 


23.7 


6.5 


Public master’s 


73.0 


53.6 


29.5 


6.3 


6.3 


9.3 


Private not-for-profit master’s 
Private not-for-profit 


62.5 


25.3 


21.6 


17.1 


14.1 


0.4 


baccalaureate 


66.0 


55.3 


4.6 


14.6 


14.3 


12.1 


Public associate’s 


69.4 


63.1 


9.1 


3.9 


4.4 


12.6 


Other 5 


57.4 


45.8 


10.7 


18.6 


9.5 


32.1 



1 Institutions also reported whether they discontinued the tenure system. Overall, less than 1 percent of institutions discontinued 
their tenure system during the past 5 years. 

“ Institutions that have downsized may have laid off faculty, replaced departing tenured faculty with nontenure track faculty, 
or not hired new faculty to replace departing faculty. 

3 All public and private not-for-profit Title IV degree-granting institutions in the 50 states and the District of Columbia. 

4 Doctoral includes research/doctoral institutions and specialized medical schools and medical centers as classified by the 2000 
Carnegie Classification. 

5 Public baccalaureate, private not-for-profit associate’s, and other specialized institutions except medical schools and medical 
centers. 

NOTE: Faculty includes all faculty and instructional staff. Standard error tables are available at 
http://iiccs.cd.gov/das/library/rcports.asp . 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary 
Faculty (NSOPF:04). 
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Appendix A — Glossary 



This glossary includes descriptions of the variables that were used in the tables of this report. The 2004 National 
Study of Postsecondary Faculty (NSOPF:04) Institution Data Analysis System (DAS) and the 2004 Integrated 
Postsecondary Education Data System (IPEDS:04) DAS were used to generate the tables in this report. Readers may 
refer to appendix B for more information on the DAS application. 

Variables are listed in the glossary index below by general topic area and in the order in which they appear in the 
tables. The glossary that follows is organized alphabetically by variable name (displayed in capital letters to the right 
of the variable label). 



Glossary Index 



NATIONAL STUDY OF POSTSECONDARY 
FACULTY INSTITUTION DATA 

Institution Survey: 2004 



Institution type X121 Q0 

Faculty represented by a union X01I12 

Full-time faculty hired in past year X03I2 

New full-time faculty who were part time X04I2 

Full-time faculty who left X05I2 

Of those who left, percent who retired X09I2 

Of those who left, percent who left for other 

reason XI 012 

Has tenure system 13 

Maximum years on tenure track 16 

Taken action related to tenure X01I7 

Offered early retirement I7E 

Replaced tenured faculty with fixed-term faculty I7D 

Changed tenure policy 17 A 

Adapted more stringent tenure policy I7B 

Downsized tenured faculty I7C 



INTEGRATED POSTSECONDARY 
EDUCATION DATA SYSTEM 

Institutional Characteristics and Student 
Charges: 2004 



Degree-granting status DEGGRANT 

Postsecondary and Title IV institution 

indicator PSET4FLG 

Carnegie Classification Code CARNEGIE 

Control of institution CONTROL 

Total enrollment ENRTOT 

Employees by Assigned Position: 2004 

Total employees EAPTOT 

Primary function and intensity of 
employee EAPRECTP 
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Variable Name 

Carnegie Classification code CARNEGIE 

Indicates the 2000 Carnegie classification code of an institution 



Doctoral Includes doctoral/research universities — extensive, doctoral/research 

universities — intensive, medical schools and medical centers 

Master’s Includes masters colleges and universities I, masters colleges and 

universities II 



Baccalaureate 



Includes baccalaureate colleges — liberal arts, baccalaureate colleges — 
general, baccalaureate/associates colleges 



Associate’s 



Includes associates colleges 



Other Includes theological seminaries and other specialized faith-related 

institutions, other separate health profession schools, schools of 
engineering and technology, schools of business and management, 
schools of art, music, and design, schools of law, teachers colleges, 
other specialized institutions, tribal colleges 



Control of institution CONTROL 

A classification of whether an institution is operated by publicly elected or appointed officials or by privately elected 
or appointed officials and derives its major source of funds from private sources. 

Public 

Private not-for-profit 
Private for-profit 



Degree -granting status DEGGRANT 

A code indicating the degree-granting status of the institution. Degree-granting institutions offer an associate’s, 
bachelor’s, master’s, doctor’s or a first-professional degree. Non-degree-granting offers certificates or other formal 
awards. 



Primary function and intensity of employee EAPRECTP 

Primary function of the employee at the institution: includes counts of employees by full- and part-time status; by 
function or occupational category; and by faculty status and tenure status. This report cites all full-time employees 
with faculty status (2101), all part-time employees with faculty status (3101), all full-time employees, primarily 
instruction, without faculty status (21 15), all part-time employees, primarily instruction, without faculty status 
(3115), all full-time employees, instruction combined with research/public service, without faculty status (2125), 
and all part-time employees, instruction combined with research/public service, without faculty status (3125). 



Total employees EAPTOT 

Total number of employees whose function type is determined by EAPRECTP on the institution’s payroll as of 
November 2004. 
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Variable Name 



Total enrollment ENRTOT 

Total men and women enrolled for credit in the fall of the academic year. 

Has tenure system 13 

Indicates whether the institution had a tenure system for full-time faculty. 

Maximum years on tenure track 16 

Indicates whether an institution limits the amount of time nontenured full-time faculty and instructional staff can 
spend on a tenure track (i.e., the maximum number of years staff can be on tenure track and not receive tenure). 

No maximum 
Less than 5 years 

5 years 

6 years 

7 years 

More than 7 years 

Changed tenure policy I7A 

Indicates whether an institution changed its tenure policy for full-time faculty between 1998 and 2003. 

Adapted more stringent tenure policy I7B 

Indicates whether an institution adapted a more stringent tenure policy for full-time faculty between 1998 and 2003. 

Downsized tenured faculty I7C 

Indicates whether an institution downsized full-time tenured faculty between 1998 and 2003. 

Replaced tenured faculty with fixed-term faculty I7D 

Indicates whether an institution replaced tenured faculty with fixed-term faculty between 1998 and 2003. 

Offered early retirement I7E 

Indicates whether an institution offered early retirement to full-time faculty between 1998 and 2003. 

Postsecondary and Title IV institution indicator PSET4FLG 

Identifies institutions who participate in the Title IV financial aid program. 
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Variable Name 

Taken action related to tenure X01I7 

Indicates whether the institution has taken any action related to tenure between 1998 and 2003. 



Faculty represented by a union X01I12 

Indicates if any full-time or part-time faculty or instructional staff employed at the institution were legally 
represented by a union. 



Full-time faculty hired in past year X03I2 

Indicates the percentage of full-time faculty who were newly hired during the 2002-03 academic year. This variable 
was calculated by dividing the number of new hires during the 2002-03 academic year by the total number of full- 
time faculty as of November 2003. 



New full-time faculty who were part time X04I2 

Indicates the percentage of new full-time faculty who were previously part time at the institution. This variable was 
calculated by dividing the number of faculty who changed from part time to full time by the total number of new 
full-time faculty hired during the 2002-03 academic year. 



Full-time faculty who left X05I2 

Indicates the percentage of full-time faculty who left between November 2002 and November 2003. This variable 
was calculated by dividing the number of full-time faculty who left by the total number of faculty in November 
2002 . 



Of those who left, percent who retired X09I2 

Indicates the percentage of full-time faculty who retired out of all faculty who left the institution. This variable was 
calculated by dividing the number of faculty who retired by the total number of faculty who left the institution. 



Of those who left, percent who left for other reason X10I2 

Indicates the percentage of full-time faculty who left for reasons other than retirement out of all faculty who left the 
institution. This variable was calculated by dividing the number of faculty who left for other reasons by the total 
number of faculty who left the institution. 



Institution type X121Q0 

This derived variable identifies the type of institution in which the respondent was employed. It was derived using 
the 2000 Carnegie Classification combined with the control (public or private not-for-profit) of the institution. The 
numeric Carnegie code is listed in parentheses after each category name. The 2000 Carnegie Classification included 
a number of changes, including a new category called "Baccalaureate/Associate’s Colleges” created to reflect the 
emergence of bachelor’s degrees being granted at 2-year institutions. These institutions are defined as colleges at 
which most of the degrees awarded are subbaccalaureate, but at least 10 percent of degrees awarded were bachelor’s 
degrees. This category includes 0.6 percent of faculty in NSOPF:04. For more information about the Carnegie 
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Variable Name 

Institution type — continued X121Q0 

Classification system adopted in 2000, see The Carnegie Classification of Institutions of Higher Education (The 
Carnegie Foundation for the Advancement of Teaching 2001). Public baccalaureate and private not-for-profit 
associate’s institutions are not in separate categories because there are too few institutions of these types to yield 
reliable estimates. 

Public doctoral Includes public institutions in the following categories: 

Doctoral/Research Universities-Extensive (15) 

Doctoral/Research Universities-lntensive (16) 

Medical schools and medical centers (52) 



Private not-for-profit doctoral Includes private not-for-profit institutions in the following categories: 

Doctoral/Research Universities-Extensive (15) 

Doctoral/Research Universities-lntensive (16) 

Medical schools and medical centers (52) 

Public master’s Includes public institutions in the following categories: 

Master’s Colleges and Universities I (21) 

Master’s Colleges and Universities II (22) 

Private not-for-profit master’s Includes private not-for-profit institutions in the following categories: 

Master’s Colleges and Universities I (21) 

Master’s Colleges and Universities II (22) 

Private not-for-profit 

baccalaureate Includes private not-for-profit institutions in the following categories: 

Baccalaureate Colleges-Liberal Arts (31) 

Baccalaureate Colleges-General (32) 

Baccalaureate/ Associate’s Colleges (33) 

Public associate’s Includes public institutions in the following category: 

Associate’s Colleges (40) 

Other Includes institutions in the following categories: 

Public baccalaureate (31, 32, and 33) 

Private not-for-profit associate’s (40) 

Theological seminaries and other specialized faith-related institutions 
(51) 

Other separate health profession schools (53) 

Schools of engineering and technology (54) 

Schools of business and management (55) 

Schools of art, music, and design (56) 

Schools of law (57) 

Teachers colleges (58) 

Other specialized institutions (59) 

Tribal colleges (60) 
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Appendix B — Technical Notes and Methodology 



Overview 

This report uses data from the fourth cycle of the National Study of Postsecondary Faculty 
(NSOPF) institution survey conducted by the U.S. Department of Education’s National Center 
for Education Statistics (NCES) in 2003-04. NSOPF was designed to provide a national profile 
of faculty and instructional staff: their professional backgrounds, responsibilities, workloads, 
salaries, benefits, and attitudes. The institution survey of NSOPF:04 provides contextual 
information (e.g., faculty tenure policies, union representation, faculty attrition) about the 
policies and practices of institutions that pertain to faculty at the institution. Previous cycles were 
conducted in 1987-88, 1992-93, and 1998-99. Additional information concerning the first three 
cycles of NSOPF is available at the NSOPF web page (http ://nces .ed. go v/surveys/nsopf/) . For 
more information on data collection procedures for NSOPF:04, please see the 2004 National 
Study of Post secondary Faculty (NSOPF:04) Methodology Report (Heuer et al. 2006). 

Sample Design 

The NSOPF:04 institution sample consisted of 1,070 eligible institutions 1 in the 50 states 
and the District of Columbia. This section provides information about the sample design for the 
2004 National Study of Postsecondary Faculty (NSOPF:04). The NSOPF:04 methodology report 
provides more detailed information. 

Institution Sample and Selection 

A single sampling stage for institutions was used for the administration of NSOPF:04. The 
institutions eligible for NSOPF:04 included those in the traditional sector of postsecondary 
education: degree-granting institutions that were eligible to participate in the federal financial aid 
programs included in Title IV of the Higher Education Act, that provide formal instructional 
degree programs of at least 2 years’ duration, that are public or private not-for-profit, and that are 
designed primarily for students who have completed the requirements for a high school diploma 
or its equivalent. NSOPF:04 does not include private for-profit or less-than-2-year institutions. 

1 Numbers are rounded to the nearest 10 to protect the confidentiality of responding institutions. Percentages cited are based on 
the original unrounded numbers. 
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The institution frame for NSOPF:04 was taken from the 2001-02 Integrated Postsecondary 
Education Data System (IPEDS) Institutional Characteristics (IPEDS-IC) file. In order to allow 
precise survey estimates for institutional sectors of interest, this universe of institutions was 
stratified based on institution control and level of degree offered. Institution control 
distinguished between public and private not-for-profit, while level of degree offered used the 
2000 Carnegie Classification system. Table B-l summarizes the number of eligible institutions 
for each of the resulting 10 primary institutional strata, based on the IPEDS-IC file. 



Table B-l. Distribution of NSOPF:04 institution universe and sample, by control and Carnegie 
Classification 



Carnegie 

Classification 


Total 




Public 




Private not-for-profit 


Universe 


Sample 


Universe 


Sample 


Universe 


Sample 


Total 


3,380 


1,080 


1,700 


680 


1,680 


400 


Doctoral 


300 


300 


190 


190 


110 


110 


Master’s 


590 


200 


270 


120 


320 


80 


Baccalaureate 


570 


160 


90 


30 


480 


130 


Associate’s 


1,180 


350 


1,030 


340 


150 


10 


Other 


730 


70 


110 


10 


620 


60 



NOTE: Universe and sample numbers are rounded to the nearest 10. Detail may not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary 
Faculty (NSOPF:04). 



The institution sampling frame for NSOPF:04 was constructed from the IPEDS-IC files. 
The institutions on the sampling frame were partitioned into 10 institutional strata based on 
institutional control, highest level of offering, and Carnegie Classification. Ultimately, a sample 
of 1,080 institutions 2 was selected using probability proportional to size (PPS) based on the 
number of faculty and students at each institution, using Chromy’s sampling algorithm (1979). 
Sample sizes and their corresponding sampling rates were established using a customized 
cost/variance optimization procedure, which aimed to identify the allocation that would 
accommodate all analytical objectives of this survey while minimizing data collection costs. 
Table B-l summarizes the distribution of the resulting sample of institutions for NSOPF:04. 
Table B-2 shows the distribution of eligible sampled institutions by institutional characteristics, 
as well as their corresponding weighted response rates. 3 



2 In order to account for the institutions that became eligible for NSOPF:04 after construction of the institution sampling frame 
from the Winter:02 IPEDS, some institutions were added to the sample. 

3 Some institutions either merged or closed after sample selection, and therefore were ineligible for NSOPF:04. 
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Table B-2. Counts of eligible and responding institutions, and response rates, by institution type: 2004 


Institution type 


Institution count 
Eligible 1 


Responding 


Weighted response 
rate (percent) 


Total 


1,070 


920 


84.2 


Public doctoral 


190 


170 


84.7 


Public master’s 


120 


110 


89.6 


Public bachelor’s 


30 


30 


100.0 


Public associate’s 


330 


290 


83.6 


Public other 


10 


10 


98.9 


Private not-for-profit doctoral 


110 


90 


83.7 


Private not-for-profit master’s 


80 


70 


79.8 


Private not-for-profit bachelor’s 


130 


110 


77.7 


Private not-for-profit associate’s 


10 


10 


86.0 


Private not-for-profit other 


60 


50 


76.2 



'Some institutions either merged or closed after sample selection and therefore were ineligible for NSOPF:04. 

NOTE: Numbers of eligible and participating institutions are rounded to the nearest 10. Detail may not sum to totals because of 
rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary 
Faculty (NSOPF:04). 



Data Quality Procedures 

Perturbation 

To protect the confidentiality of NCES data that contain information about specific 
individual cases, NSOPF:04 data were subject to perturbation procedures to minimize disclosure 
risk. Perturbation procedures, which have been approved by the NCES Disclosure Review 
Board, preserve the central tendency estimates, but may result in slight increases in nonsampling 
errors. 

Imputation of Missing Data 

All NSOPF:04 institution variables with missing data used in this report, as well as those 
included in the related Data Analysis System (DAS) release, have been imputed. The imputation 
process for the missing data from the institution questionnaire involved similar steps to those 
used for imputation of the faculty data. The missing data for variables were imputed using the 
weighted sequential hot-deck method. For this purpose, variables were partitioned into 
unconditional and conditional groups. The first group (unconditional) consisted of variables that 
apply to all responding institutions, while the second group (conditional) consisted of variables 
that apply to only a subset of the institutions. The unconditional variables were sorted by 
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percentage missing and then imputed in the order from the lowest percentage missing to the 
highest. The conditional group was partitioned into three subgroups based on the level of 
conditionality for each variable, and then imputed in that order. The imputation classes for both 
unconditional and conditional variables were defined in terms of the institution sampling stratum 
with implicit sorting variables based on the number of full-time and part-time faculty members. 
Within each subgroup, variables were imputed in the order from the lowest to the highest 
percentage missing. Overall, 87 variables from the institution questionnaire were imputed, 
including 58 variables with 1 to 5 percent imputed, 15 variables with 5 to 10 percent imputed, 11 
variables with 10 to 15 percent imputed, and 3 variables with more than 15 percent imputed 
(none of these 3 variables are used in this report). 

Weighting 

All estimates in this report are weighted to represent the target population described in the 
sample design section. The weights compensate for the unequal probability of selection of 
institutions in the NSOPF sample. See table B-3 for a complete list of NSOPF:04 weights. The 
weights also adjust for frame multiplicity, 4 nonresponse, and poststratification. The steps taken 
to calculate the study weights, WTB00, for the sample institutions data are summarized next. 



Base weights 

The institution sampling frame for NSOPF:04 included a total of 3,380 eligible units, 
detailed composition of which is provided in section 2.1.1 of the 2004 National Study of 
Postsecondary Faculty (NSOPF:04) Methodology Report (Heuer et al. 2006). Reflecting the 
probability proportional to size scheme of sample selection, the probability of selection for 
institution i in stratum r was calculated by: 




n r = sample size for stratum r, 

Sri = composite measure of size for institution i in stratum r, and 



4 It was determined after institution sample selection that in some cases, either an institution had merged with another institution, 
or faculty lists for two or more campuses were submitted as one combined faculty list. In these instances, the institution weights 
were adjusted for the joint probability of selection. 
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Table B-3. Summary of weight components for NSOPF:04 



Weight name 


Type of weight 


Associated file 


Adjustments 


Purpose 


WTB00 


Study weight 


Institution DAS 
and ECB 


Institution’s probability of selection, adjusted 
- for institutions with multiple chances of selection 


Final weight for computing point estimates 
from the institution survey data; represented 


WTB01-WTB64 


Replicate weights 




- to match published totals to ensure population 
coverage 


population is 2- and 4-year degree-granting 
public and private not-for-profit institutions 
in the 50 states and the District of Columbia. 


WTA00 


Study weight 


Faculty DAS 
and ECB 


Faculty’s probability of selection adjusted 
- for faculty members who taught at more than one 


Final weight for computing point estimates 
from the faculty survey data; represented 


WTA01-WTA64 


Replicate weights 




institution 

- for nonresponding faculty members with unknown 
eligibility 

- to compensate for nonresponding faculty members 

- to match known published totals to ensure 
population coverage 


population is faculty and instructional staff 
at institution population described above. 
No institution survey responses may be 
analyzed with this weight. 


WTC00 


Contextual weight 


Faculty ECB 
only 


Faculty’s probability of selection adjusted 
- for faculty members who taught at more than one 


Final weight for computing point estimates 
from the faculty survey data; represented 


WTC01-WTC64 


Replicate con- 
textual weights 




institution 

- for nonresponding faculty members with unknown 
eligibility 

- to compensate for nonresponding faculty members 

- to match known published totals to ensure 
population coverage 

- for whether the respondent was at an institution 
with a completed institution survey 


population is faculty and instructional staff 
at institution population described above. For 
use when institution survey responses are 
merged with the faculty data for analysis. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary Faculty (NSOPF:04). 
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S r+ = composite measure of size for all institutions in stratum r. 

The initial sample consisted of 1,220 institutions. However, this sample was reduced to a 
subsample of institutions, since a smaller sample was deemed adequate to secure all precision 
requirements of NSOPF:04. Therefore, the sampling weight for institution i in stratum r was 
calculated as a function of its initial and subsequent selection probabilities. With R r representing 
the subsampling rate in stratum r, the sampling weight for the / - 1 h institution in that stratum was 
calculated by: 



WTl 



ri 



1 1 

— x — 

K ri R r 



It should be noted that during the sample refreshing step, institutions were added to the 
sample of institutions, resulting in a total sample of 1,080 institutions for NSOPF:04. 



Adjustment for institution multiplicity 

During the institution recmitment and faculty list sampling stages, a number of institutions 
were identified that had two or more records listed on the Integrated Postsecondary Education 
Data System (IPEDS). In some cases this was caused by institutions that had recently merged, 
while in other cases the sample institution had sent a single faculty list covering multiple 
campuses. For sampling purposes, combined faculty lists that could not be separated were treated 
as merged institutions and identified under a single IPEDS ID for purposes of tracking survey 
results. 



For institutions with more than one chance of selection, a multiplicity adjustment factor 
was calculated by estimating, as if the selections were independent, the probability that each 
record could be selected. Consequently, when an institution had n chances of selection, its 
probability of selection was calculated by: 



( n \ 



if = i -n&-w,)] 



V 1=1 J l'=l 

Next, a multiplicity adjustment factor for the i-th sample institution was calculated by: 

1 



WT2 ; 



( n 

•Ik. 

V i= 1 J 



xW7T, 
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If the given institution did not require such adjustment, its multiplicity adjustment factor 
was set to unity. This way, the product of WT1 and WT2 equals the reciprocal of the resulting 
multiple chance of selection for the institutions with positive multiplicity, and equals WT1 for all 
other institutions. 

Nonresponse adjustment 

For calculation of the analysis weights for institutions responding to the institution 
questionnaire, an institution (questionnaire) level nonresponse adjustment factor ( WT3 ) was 
constructed within each of the 10 sampling strata using a Generalized Exponential Model (GEM) 
(Folsom and Singh 2000). 

Poststratification adjustment 

A set of poststratification adjustment factors ( WT4 ) was calculated for the 920 institutions 
responding to the questionnaire using the GEM program. Specifically, nonresponse-adjusted 
weights for these institutions were ratio-adjusted to the counts of institutions obtained from the 
sampling frame. Moreover, an additional adjustment factor was calculated to ensure that 
weighted counts of faculty obtained from the institution survey data would coincide with those 
obtained from the faculty survey data. With this last adjustment factor computed (WT5), the final 
analysis weight for each responding institution (WTB00) was calculated by: 

WTB00 = WT1 x WT2 x WT3 x WT4 x WT5 

The sample of institutions from which faculty were selected overlaps with, but is not 
identical to, the sample of institutions that completed the institution survey. With a license to use 
the restricted data, institution survey information can be merged onto the faculty data so that 
faculty outcomes can be compared across institutions according to variables from the institution 
survey. A contextual weight for the faculty data (WTC00) was developed for use in such 
situations — that is, when using institution survey items in analyses of faculty. 

Quality of Estimates 

Survey weights are computed with the goal of removing any bias that might result due to 
differential nonresponse and undercoverage. In order to measure the efficacy of bias-reducing 
adjustments, a series of analyses were conducted at the unit and item levels. In the subsequent 
sections, highlights of these analyses are summarized. 
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Unit response rates and bias analysis 

The institution (unit) level response rates for NSOPF:04 institutions were reviewed, overall 
and within each sampling stratum. As shown in table B-2, 920 of the 1,070 eligible sample 
institutions were respondents (84.2 weighted percent). The weighted response rates by type of 
institution ranged from 76.2 for private not-for-profit other institutions to 100 percent for public 
bachelor’s institutions. In all, the institution weighted response rate was below 85 percent for six 
of the ten types of institutions: public doctoral, public associate’s, private not-for-profit doctoral, 
private not-for-profit master’s, private not-for-profit bachelor’s, and private not-for-profit other. 
When the weighted response rates were below 85 percent, an evaluation was conducted to assess 
the potential magnitude of nonresponse bias. 

The steps for nonresponse bias analysis included estimating the nonresponse bias and 
testing (adjusting for multiple comparisons) to determine if the bias was significant at the 5 
percent level. Second, nonresponse adjustment factors were computed to significantly reduce or 
eliminate nonresponse bias for variables included in the corresponding models. Third, after the 
weights were computed, any remaining bias was estimated and statistical tests were performed to 
determine their significance. 

The bias in an estimated mean based on respondents, y R , is the difference between this 
estimate and the target parameter, //, which is the mean that would result if a complete census of 
the target population was conducted and all units responded. This bias can be expressed as 
follows: 



B(y R ) = y R -p 

However, for variables that are available from the frame, Ll can be estimated by ft without 
sampling error, in which case the bias in y R can then be estimated by: 

B(y R ) = y R -fi 

Moreover, an estimate of the population mean based on respondents and nonrespondents can be 
obtained by: 

fi = ^-v)y R +ny NR 

where;) is the weighted unit nonresponse rate, based on weights prior to nonresponse adjustment. 
Consequently, the bias in y R can then be estimated by: 

B(y R ) = n{y R -y NR ) 
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That is, the estimate of the nonresponse bias is the difference between the mean for respondents 
and nonrespondents multiplied by the weighted nonresponse rate, using the institution-level 
design weight prior to nonresponse adjustment. 

This procedure parallels the one carried out for nonresponse bias analysis of the faculty 
data, but the institution data included a smaller sample size and a smaller number of attributes 
known for both respondents and nonrespondents. For the six institution types with nonresponse 
rates of 15 percent or higher, Table B-4 shows the percent significant nonresponse bias 5 before 
weight adjustments (third row in table) and after weight adjustments (final row in table). The 
results show reduced nonresponse bias overall and for all institution types. 



Table B-4. Summary of institution nonresponse bias analysis for institution types with nonresponse rates 
of 15 percent or higher: 2004 



Private 











Private 


Private 


not-for- 


Private 








Public 


not-for- 


not-for- 


profit 


not-for- 




All 


Public 


asso- 


profit 


profit 


bach- 


profit 


Nonresponse bias statistics 


institutions 


doctoral 


ciate’s 


doctoral 


master’ s 


elor’s 


other 



Before weight adjustments 



Mean estimated relative bias 


0.06 


0.06 


0.12 


0.08 


0.08 


0.11 


0.12 


Median estimated relative bias 


0.04 


0.05 


0.09 


0.06 


0.07 


0.13 


0.09 


Percent significant bias 


5.26 


6.25 


18.75 


12.50 


6.25 


13.33 


# 


After weight adjustments 


Mean estimated relative bias 


0.20 


0.09 


0.18 


0.15 


0.29 


0.23 


0.53 


Median estimated relative bias 


0.12 


0.07 


0.11 


0.13 


0.25 


0.18 


0.42 


Percent significant bias 


2.63 


6.25 


12.50 


6.25 


# 


6.67 


# 



# Rounds to zero. 

NOTE: The percent significant bias is calculated as the ratio of the number of estimates with significant bias to 
the total number of estimates involved in the given analysis. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of 
Postsecondary Faculty (NSOPF:04). 



Finally, note that such unit-level bias analyses were not carried out when computing the 
institution component for the faculty analysis weights, since the weighted response rate for 
institutions providing lists of faculty and instructional staff exceeded 85 percent for all sectors. 



5 The percent significant bias is calculated as the ratio of the number of estimates with significant bias to the total number of 
estimates involved in the given analysis. That is. 

Number of estimates with significant bias 

Percent Significant Bias = xlOO 

Number of estimates investigated 
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See chapter 3 of the 2004 National Study of Postseconclary Faculty (NSOPF:04) Methodology 
Report for more information on institution list collection (Heuer et al. 2006). 

Item-level bias analysis 

Item response rates ( IRR ) are calculated as the ratio of the number of respondents for 
whom an in-scope response was obtained (f for item x) to the number of respondents who are 
asked to answer that item. The number asked to answer an item is the number of unit level 
respondents (7) minus the number of respondents with a valid skip item for item x (V A ). 

r 

IRR x = 

I-V x 

An institution is defined to be an item respondent for an interview variable if that 
institution has data for that variable, including logical imputation and recoding based on 
information obtained elsewhere. Overall, the rates of missing data were low to moderate: 
nonresponse was less than 15 percent for most items. 

A nonresponse bias analysis was conducted for those variables with a response rate of less 
than 85 percent (nonresponse above 15 percent) overall or for any institutional stratum. Table B- 
5 presents item response rates for items with less than 85 percent response rate for any 
institutional stratum before weight adjustments. Table B-6 summarizes the nonresponse bias 
analysis for qualifying variables used in this report. The nonresponse bias was estimated for 
variables known for both respondents and nonrespondents and tested (adjusting for multiple 
comparisons) to determine if the bias was significant at the .05 level. The percentage significant 
reflects the ratio of the number of biased estimates to the number of biased and unbiased 
estimates. Although there was 25 percent significant bias for public bachelor’s institutions, the 
mean estimated bias was small (0.01), and its impact on the accuracy on estimates was judged to 
be minimal. Because there are so few public bachelor’s institutions in the universe, public 
bachelor’s institutions and faculty within them are not shown separately in figure 1 or tables 1-5 
of this report, but are included in the “Other” category for these tables and figure. 

Standard Errors 

In order to facilitate computation of standard errors for both linear and nonlinear statistics, 
a vector of bootstrap sample weights has been added to the analysis file. These weights are zero 
for units not selected in a particular bootstrap sample; weights for other units are inflated for the 
bootstrap subsampling. The initial analytic weights for the complete sample are also included for 
the purposes of computing the desired estimates. The vector of replicate weights allows for 



30 





Table B-5. Institution item response rates for items with less than 85 percent response rate for any institution type before weight adjustments, 
overall and by institution type: 2004 





AU. 






Public 








Private not-for-profit 




Variable 


institu- 

tions 


Doctor’ s 


Master’ s 


Bach- 

elor’s 


Asso- 

ciate’s 


Other 


Doctor’s 


Master’s 


Bach- 

elor’s 


Asso- 

ciate’s 


Other 


Full-time tenure: changed tenure policy 


96.3 


97.6 


94.8 


70.6 


95.9 


100.0 


98.4 


98.4 


95.9 


100.0 


100.0 


Full-time tenure: more stringent tenure 
standards 


96.3 


97.6 


94.8 


70.6 


95.9 


100.0 


98.4 


98.4 


95.9 


100.0 


100.0 


Full-time tenure: downsized tenured faculty 


96.3 


97.6 


94.8 


70.6 


95.9 


100.0 


98.4 


98.4 


95.9 


100.0 


100.0 


Full-time tenure: replaced tenured with 
fixed term 


96.1 


97.6 


94.8 


70.6 


95.9 


100.0 


98.4 


98.4 


94.7 


100.0 


100.0 


Full-time tenure: offered early retirement 


96.3 


97.6 


94.8 


70.6 


95.9 


100.0 


98.4 


98.4 


95.9 


100.0 


100.0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary Faculty (NSOPF:04). 






Table B-6. Summary of institution item nonresponse bias analysis, overall and by institution type: 2004 











Public 








Private not-for-profit 












Bach- 


Asso- 








Bach- 


Asso- 




Variable All faculty Doctor’s 


Master’s 


elor’s 


ciate’s 


Other 


Doctor’s 


Master’s 


elor’s 


ciate’s 


Other 
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Full-time tenure: more stringent tenure standards 
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Full-time tenure: downsized tenured faculty 
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Full-time tenure: replaced tenured with fixed term 
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Full-time tenure: offered early retirement 
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t Not applicable. 

# Rounds to zero. 

NOTE: The percent significant bias is calculated as the ratio of the number of estimates with significant bias to the total number of estimates involved in the given analysis. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary Faculty (NSOPF:04). 
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computing additional estimates for the sole purpose of estimating a variance. Assuming B sets of 
replicate weights, the variance of any estimate, 9 , can be estimated by replicating the estimation 
procedure for each replicate and computing a simple variance of the replicate estimates; i.e., 

±(o b -of 

Var{0) = H 

B 

where 9 b is the estimate based on the b ih replicate weight and B is the number of replicates. Once 
the replicate weights are provided, this estimate can be produced by most survey software 
packages (e.g., SUDAAN [RTI 2004]). 

The replicate weights were produced using a methodology and computer software 
developed by Kaufman (2004). This methodology allows for finite population correction factors 
at two stages of sampling. Application of this method incorporated the finite population 
correction factor at the first stage only where sampling fractions were generally high. At the 
second stage, where the sampling fraction was generally low, the finite population correction 
factor was set to 1.0. 

Data Analysis System 

The estimates presented in this report were produced using the NSOPF:04 Institution Data 
Analysis System (DAS). The DAS application on the Web makes it possible for users to specify 
and generate their own tables. With the DAS, users can replicate or expand upon the tables 
presented in this report. In addition to the table estimates, the DAS calculates proper standard 
errors 6 and weighted sample sizes for these estimates. For example, table B-7 contains standard 
errors that correspond to estimates in table 2 of the report. If the number of valid cases is too 
small to produce a reliable estimate (fewer than 30 cases), the DAS prints the message “low n” 
instead of the estimate. All standard errors for estimates presented in this report can be viewed at 
http ://nces .ed. gov/das/library/reports . asp . In addition to tables, the DAS will also produce a 
correlation matrix of selected variables to be used for linear regression models. Included in the 
output with the correlation matrix are the design effects (DEFTs) for each variable in the matrix. 
Since statistical procedures generally compute regression coefficients based on simple random 
sample assumptions, the standard errors must be adjusted with the design effects to take into 
account the stratified sampling method used in the survey. 



6 The NSOPF:04 sample is not a simple random sample, and therefore, simple random sample techniques for estimating 
sampling error cannot be applied to these data. The DAS takes into account the complexity of the sampling procedures and 
calculates standard errors appropriate for such samples. 
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Table B-7. Standard errors for table 2: Percentage of newly hired full-time faculty and percentage of new 
full-time faculty who were part-time faculty at that same institution in November 2002, by 
institution type: Fall 2003 


Institution type 


Percent of full-time faculty 
hired within the past 
year from outside 
the institution 


Percent of new full-time 
faculty who were 
previously part time 
at the same institution 


All institutions 


0.21 


0.61 


Public doctoral 


# 


# 


Private not-for-profit doctoral 


0.07 


0.07 


Public master’s 


0.73 


1.35 


Private not-for-profit master’s 


0.70 


1.95 


Private not-for-profit baccalaureate 


1.66 


2.92 


Public associate’s 


0.69 


2.25 


Other 


1.21 


2.29 



# Rounds to zero. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2004 National Study of Postsecondary 
Faculty (NSOPF:04). 



The DAS can be accessed electronically at http://nces.ed.gov/das/ . For more information 
about the NSOPF:04 Data Analysis System, contact: 

Aurora D’Amico, Postsecondary Studies Division 

National Center for Education Statistics 

1990 K Street NW 

Washington, DC 20006-5652 

(202) 502-7334, Aurora.D’ Amico@ed.gov 

Statistical Procedures 

The descriptive comparisons were tested in this report using Student’s t statistic. 
Differences between estimates are tested against the probability of a Type I error, 7 or 
significance level. The significance levels were determined by calculating the Student’s t values 
for the differences between each pair of means or proportions and comparing these with 
published tables of significance levels for two-tailed hypothesis testing. 

Student’s t values may be computed to test the difference between estimates with the 
following formula: 



7 A Type I error occurs when one concludes that a difference observed in a sample reflects a true difference in the population 
from which the sample was drawn, when no such difference is present. 
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Ei — Ei 




where Ej and Ei are the estimates to be compared and set and sei_ are their corresponding 
standard errors. This formula is valid only for independent estimates. When estimates are not 
independent, a covariance term must be added to the formula: 




where r is the correlation between the two estimates. 8 This formula is used when comparing two 
percentages from a distribution that adds to 100. If the comparison is between the mean of a 
subgroup and the mean of the total group, the following formula is used: 




where p is the proportion of the total group contained in the subgroup. The estimates, standard 
errors, and correlations can all be obtained from the DAS. 

There are hazards in reporting statistical tests for each comparison. First, comparisons 
based on large t statistics may appeal’ to merit special attention. This can be misleading since the 
magnitude of the t statistic is related not only to the observed differences in means or percentages 
but also to the number of respondents in the specific categories used for comparison. Hence, a 
small difference compared across a large number of respondents would produce a large 1 
statistic. 

A second hazard in reporting statistical tests is the possibility that one can report a “false 
positive” or Type I error. In the case of a t statistic, this false positive would result when a 
difference measured with a particular sample showed a statistically significant difference when 
there is no difference in the underlying population. Statistical tests are designed to control this 
type of error, denoted by alpha. The alpha level of .05 selected for findings in this report 
indicates that a difference of a certain magnitude or larger would be produced no more than one 
time out of twenty when there was no actual difference in the quantities in the underlying 
population. When we test hypotheses that show t values at the .05 level or smaller, we treat this 



8 U.S. Department of Education, National Center for Education Statistics, A Note from the Chief Statistician, no. 2, 1993. 
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finding as rejecting the null hypothesis that there is no difference between the two quantities. 
Failing to detect a difference, however, does not necessarily imply the values are the same or 
equivalent. 
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